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Abstract 


A  proof  of  concept  (POC)  assessment  was  conducted  on  the  NCOT  facility  to  determine  its 
ability  to  support  Test  and  Evaluation  requirements  for  real-time,  human-system  performance 
measurement.  The  goal  was  to  look  at  the  NCOT  capability  to  develop,  build  and  run  scenarios 
and  to  capture  a  range  of  performance  data  from  individuals  and  small  teams.  In  general,  it  was 
concluded  that  for  the  most  part  the  NCOT  facility  provided  the  means  for  building  and  running 
scenarios,  however  software  modifications  would  be  need  to  better  support  T&E  data  capture  and 
analysis  requirements. 


Resume 


Le  NCOT  faisait  Tobjet  d’une  validation  de  principe  (VP)  pour  determiner  sa  capacite  a  soutenir 
les  exigences  d’essai  et  d’evaluation  relatives  a  la  mesure  du  rendement  homme-machine  en 
temps  reel.  Le  but  etait  de  verifier  sa  capacite  a  elaborer,  a  batir  et  a  executer  des  scenarios  ainsi 
qu’a  recueillir  des  donnees  sur  le  rendement  de  personnes  et  de  petites  equipes.  Selon  les 
constats,  le  NCOT  permet  surtout  de  batir  et  d’executer  des  scenarios,  car,  pour  mieux  recueillir 
et  analyser  des  donnees,  il  faudrait  modifier  son  logiciel. 
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Executive  Summary 


This  report  represents  the  next  step  in  an  ongoing  process  to  provide  systematic  human  factors 
support  to  the  COMDAT  program,  in  terms  of  developing  a  trial  plan  and  detailed  methods  and 
measures  to  evaluate  the  impact  and  effectiveness  of  new  decision  support  technologies  to  the 
Ops  Room  of  the  Halifax  class  ships. 

In  previous  reports  we  have  outlined  potential  data  collection  sites  and  have  developed  a  detailed 
trial  plan  specifying  what  C2  processes  to  measure,  and  a  number  of  methods  to  capture  the 
required  data  for  test  and  evaluation  (T&E)  purposes.  The  next  step  in  the  process  has  been  to 
assess  the  capability  of  the  NCOT  facility  to  support  real  time  data  collection  and  analysis 
through  a  Proof  of  Concept  (POC)  trial. 

The  trial  involved  the  development  and  building  of  a  complex  scenario  that  comprised 
background  events  in  the  air  and  on  the  surface  from  commercial  and  other  traffic. 

Superimposed  upon  the  background  scenario  were  a  number  of  potential  threat  events  arising  out 
of  the  specific  geo-political  circumstances. 

The  scenario  was  then  run  in  real  time  in  NCOT  using  a  small  team  of  Navy  supplied  SMEs  to 
play  the  roles  of  the  RTl,  TrackSup,  SWC  and  ORO.  We  then  assessed  the  ability  of  NCOT  to 
support  a  wide  range  of  requirements  in  the  actual  running  of  the  scenario.  These  included  the 
ability  (1)  to  deliver  information  to  the  team  from  within  the  scenario  and  from  T&E  personnel 
playing  other  Ops  Room  roles;  (2)  to  capture  ORO  actions,  team  communications  and  CCS 
screen  content  (3)  to  time  and  log  events  of  interest  for  T&E  analysis  and  (4)  to  capture  a  video 
and  audio  record  of  team  interaction.  Subsequently,  we  then  assessed  the  NCOT  capability  to 
playback  the  recorded  scenario  events  for  analysis  for  T&E  purposes  and  to  allow  appropriate 
management  of  trial  data. 

We  concluded  that  NCOT  provides  good  support  for  scenario  building,  running  and  data  capture. 
Although  several  suggestions  for  system  improvements  were  made  to  make  these  processes  more 
efficient,  reliable  and  effective  for  future  data  collection  trials.  In  terms  of  data  management, 
playback  and  analysis,  we  identified  a  number  of  shortcomings  with  the  NCOT  system  to  support 
the  unique  and  specific  requirements  for  T&E  purposes.  This  was  not  surprising  since  NCOT 
has  been  conceived  and  designed  as  a  Navy  training  facility  rather  than  a  T&E  test  bed.  A 
number  of  practical  solutions,  both  hardware  and  software  based,  were  suggested  to  overcome 
the  existing  limitations  of  the  system  for  playback  and  analysis  purposes. 

It  was  concluded  that  if  such  solutions  and  improvements  can  be  implemented,  the  NCOT  facility 
will  be  a  suitable  environment  for  conducting  T&E  trials  using  small  Ops  Room  teams,  with  a 
view  to  measuring  and  assessing  key  Ops  Room  C2  processes  that  may  be  impacted  by 
COMDAT  technology. 
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Sommaire 


Grace  au  present  rapport,  nous  franchissons  une  autre  etape  du  processus  permanent  visant  a 
fournir  un  soutien  en  matiere  de  facteurs  humains  systematiques  au  programme  COMDAT.  Ce 
soutien  porte  essentiellement  sur  I’elaboration  d’un  plan  d’essai  ainsi  que  de  methodes  et  de 
mesures  exhaustives  pour  evaluer  les  incidences  et  I’efficacite  de  la  nouvelle  technologie  d’aide 
aux  decisions  du  C  Op  des  navires  de  la  classe  Halifax. 

Dans  les  rapports  precedents,  nous  avons  releve  des  sites  potentiels  de  collecte  de  donnees  et 
elabore  un  plan  d’essai  detaille,  indiquant  les  processus  de  C2  a  mesurer,  et  des  methodes  pour 
recueillir  les  donnees  necessaires  a  I’essai  et  a  revaluation  (E  et  E).  Nous  devions  ensuite 
evaluer,  au  moyen  d’un  essai  de  validation  de  principe  (VP),  la  capacite  du  NCOT  a  soutenir  la 
collecte  et  1’ analyse  de  donnees  en  temps  reel. 

L’essai  comprenait  I’elaboration  et  I’edification  d’un  scenario  complexe  comportant,  en  toile  de 
fond,  des  evenements  aeriens  et  terrestres  lies  au  trafic  commercial  et  autres.  S’y  ajoutaient  un 
certain  nombre  de  menaces  potentielles  decoulant  de  circonstances  geopolitiques  precises. 

Nous  avons  execute  le  scenario  en  temps  reel  dans  le  NCOT  avec  quelques  specialistes  en  la 
matiere,  pretes  par  la  Marine,  qui  jouaient  les  roles  de  RTl,  de  superviseur  de  piste,  de  CAD  et 
d’O  C  Op.  Nous  avons  evalue  la  capacite  du  NCOT  a  soutenir  toute  une  gamme  d’exigences, 
pendant  I’execution  reelle  du  scenario,  dont  celles  1)  de  conununiquer  a  I’equipe  I’information 
tiree  du  scenario  et  provenant  du  personnel  d’E  et  E  dans  le  role  d’autres  C  Op,  2)  d’enregistrer 
les  actions  de  I’O  C  Op,  les  communications  de  I’equipe  et  le  contenu  de  I’ecran  du  CCOS,  3) 
d’enregistrer  les  evenements  d’interet  pour  I’analyse  d’E  et  E  ainsi  que  I’heure  a  laquelle  ils  se 
sont  produits  et  4)  d’enregistrer  I’interaction  de  I’equipe  (video  et  audio).  Puis,  nous  avons 
evalue  la  capacite  du  NCOT  a  reconstituer  les  evenements  enregistres  a  des  fins  d’analyse  dans  le 
cadre  de  I’E  et  E  et  a  permettre  la  gestion  convenable  des  donnees  d’essai. 


Nous  avons  conclu  que  le  NCOT  soutient  bien  I’edification  et  I’execution  de  scenarios  ainsi  que 
la  collecte  de  donnees.  Neanmoins,  nous  avons  emis  plusieurs  suggestions  pour  ameliorer  le 
systeme  ainsi  que  I’efficacite,  la  fiabilite  et  I’efficience  des  processus  en  vue  des  prochains  essais 
de  collecte  de  donnees.  Quant  a  la  gestion  des  donnees,  a  la  reconstitution  et  a  1’ analyse,  nous 
avons  releve  des  lacunes  a  I’egard  des  exigences  uniques  et  precises  d’E  et  E.  Rien  de 
surprenant,  car  le  NCOT  a  ete  congu  en  tant  qu’equipement  de  formation  de  la  Marine  et  non  de 
banc  d’essai  pour  I’E  et  E.  Nous  avons  suggere  des  solutions  pratiques,  materielles  et  logicielles, 
pour  repousser  les  limites  actuelles  du  systeme  en  matiere  de  reconstitution  et  d’analyse. 

En  mettant  en  oeuvre  ces  solutions  et  ces  ameliorations,  le  NCOT  constituera  un  environnement 
adequat  pour  mener  des  E  et  E  avec  de  petites  equipes  de  C  Op  et  ainsi  mesurer  et  evaluer  les 
principaux  processus  de  C2  du  C  Op  qui  peuvent  etre  touches  par  la  technologie  COMDAT. 


Humansystems  Incorporated 


NCOT :  Proof  of  Concept 


Page  5 


P517839.PDF  [Page:  8  of  39] 


Table  of  Contents 


Abstract . 3 

Resume . 3 

Executive  Summary . 4 

Sommaire . 5 

Table  of  Contents . 6 

1.  Introduction . 8 

2.  Goals  of  the  Proof  of  Concept . 9 

3.  Method  used . 10 

4.  Results . 11 

4.1  Scenario . 11 

4.1.1  Scenario  development . 11 

4.1.2  Did  the  scenario  unfold  in  real-time  in  the  manner  intended? . 1 1 

4.1.3  Were  the  mission  foreground  threat  events  suitable? . 11 

4. 1 .4  Was  the  scenano  suitable  for  producing  the  required  level  of  workload? . 1 1 

4.2  General  logistics  for  running  the  scenario . 12 

4.2. 1  Time  required  for  set-up  and  preparation . 12 

4.2.2  Personnel  support . 13 

4.2.3  Robustness  &  reliability  of  system  hardware  &  software  during  play . 1 5 

4.2.4  Ability  to  record  additional  audio  and  video . 1 6 

4.2.5  Suitability  and  adequacy  of  communications  among  scenario  players . 16 

4.2.6  Ability  to  support  specific  T&E  requirements . 16 

4.2.6. 1  Control  of  game  entities  in  real  time . 16 

4. 2. 6. 2  Communications  from  T&E/actors  to  Ops  Room  Team . 18 

4. 2. 6. 3  Consistency  of  scenario  repetition  across  trials . 18 

4. 2. 6. 4  Support  for  collecting  specific  MOPs . 18 

4. 2. 6. 5  Recording  of  trial  data . 20 

Humanvy^tew-y  Incorporated®  NCOT:  Proof  of  Concept  Page  6 


P517839.PDF  [Page:  9  of  39] 


4. 2. 6. 6  Playback  and  analysis  of  NCOT  data  files . 21 

5 .  Conclusions . 27 

6 .  Recommendations . 28 

6. 1  Software  and  system  improvements . 28 

6.2  Data  capture  and  analysis . 29 

6.3  Personnel  requirements  to  support  T&E . 29 

6.4  Scenario  modification . 30 

6.5  Other  issues . 30 

7.  References . 31 

Annex  A:  Scenario  Description . 2 

General  Situation . 2 

Rules  of  Engagement: . 3 

Identification  Criteria: . 3 


Ylumansystems  Incorporated® 


NCOT:  Proof  of  Concept 


Page  7 


P517839.PDF  [Page:  10  of  39] 


1.  Introduction 


The  work  reported  in  this  Technical  Memorandum  arises  from  a  contract  to  Humaus^’^rmi' 
Incorporated*  (HSI*  )  from  the  Defence  and  Civil  Institute  of  Environmental  Medicine  (DCIEM). 
The  Scientific  Authority  (SA)  for  the  work  is  Carol  McCann.  The  work  continues  an  ongoing 
program  of  test  and  evaluation  (T&E)  whose  goal  is  to  create  reliable  and  valid  measures  of  ORO 
performance  in  the  Halifax  class  Ops  Room,  with  a  view  to  assessing  the  impact  of  future 
decision  support  technologies. 

This  Technical  Memorandum  summarises  the  outcome  of  the  Test  and  Evaluation  (T&E)  Proof 
of  Concept  (POC)  Trial  at  the  NCOT  facility  that  was  conducted  in  accordance  with  the  T&E 
trial  plan  outlined  in  reference  1 .  The  POC  was  conducted  by  HSI  ®  team  members  during  a 
visit  to  NCOT  on  March  11-13  2002. 

This  memorandum  is  organised  into  the  following  sections; 

■  Goals  of  the  POC 

■  Method  used 

■  Results  of  the  assessment 

■  Conclusions 

■  Recommendations 
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2.  Goals  of  the  Proof  of  Concept 


The  overall  objective  was  to  assess  the  suitability  of  the  NCOT  facility  to  provide  the  required 
environment  for  future  T&E  trials  to  gather  human-system  performance  data  on  small  Ops  Room 
teams.  The  intent  of  the  trials  would  be  to  generate  reliable  performance  data  on  a  variety  of  C2 
tasks  using  a  simulation  of  the  existing  Ops  Room  equipment.  A  successful  outcome  would  serve 
two  goals.  First,  it  would  demonstrate  that  meaningful  and  reliable  C2  MOPs  could  indeed  be 
collected.  Second,  the  data  obtained  would  provide  a  basis  for  assessing  the  future  impact  of 
new  COMDAT  technologies. 

The  specific  goals  of  the  POC  were  to  investigate  the  suitability  of  NCOT  to  support  T&E  trials 
in  the  following  areas. 

Scenario 

■  Building  the  scenario 

■  Testing  that  the  scenario  unfolded  in  real-time  in  the  manner  intended 

■  Evaluating  whether  the  foreground  threat  events  were  suitable 

■  Assessing  the  suitability  of  the  scenario  for  producing  the  required  level  of  ORO  and 
team  workload 

General  logistics  for  running  the  scenario 

■  Time  required  for  set-up  and  preparation 

■  Personnel  support:  NCOT  technical  support.  Navy  SMEs,  T&E  team  (HSI* ). 

■  Robustness  and  reliability  of  the  system  hardware  and  software  during  the  scenario 

■  Ability  to  record  additional  audio  and  video 

■  Suitability  and  adequacy  of  communications  among  scenario  players 

■  Suitability  and  adequacy  of  data  supplied  to  scenario  players 

■  Personnel  support  requirements: 

Ability  to  support  specific  T&E  requirements 

■  Control  of  game  entities  in  real  time 

■  Communications  from  T&E/actors  to  Ops  Room  Team 

■  Consistency  of  scenario  repetition  across  trials 

■  Recording  of  trial  data 

■  Playback  and  analysis  of  NCOT  data  files 
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3.  Method  used 

A  scenario  was  prepared  ahead  of  time,  details  of  whicli  are  provided  in  Annex  A.  The  scenario 
was  designed  to  create  a  moderate  level  of  workload  for  the  team  and  comprised  background 
surface  and  air  traffic  overlaid  with  surface  and  air  threat  events.  The  Ops  Room  team  required 
to  run  the  scenario  comprised  an  RTl,  Track  Sup,  SWC  and  ORO.  HST  staff  (ex  navy  ORO 
SMEs)  provided  additional  information  to  the  team  that  would  normally  come  from  the  CO, 

OOW,  CANEWS  and  any  other  relevant  sources.  The  NCOT  facility  manager  provided  full 
time  support  to  the  team  in  setting  up,  running  and  analysing  the  scenario. 

The  structure  of  the  POC  trial  was  as  follows: 

Day  1; 

■  Play  and  review  scenario  in  real  time  and  correct  any  problems 

■  Configure  workstations  for  the  Ops  room  team 

■  Set  up  T&E  logistics  for  audio  and  video  recording 
•  Rehearse  roles  and  control  of  game  entities 

■  Review  and  rehearse  requirements  from  NCOT  support  staff 

Day  2: 

■  Brief  Navy  SMEs  participating  as  subjects  on  purpose  of  POC,  project. 

■  Review  mission  background  for  scenario  with  Navy  participants. 

■  Familiarise  Navy  participants  with  Workstations  and  Comms 

■  Create  necessary  scenario  overlays  by  RTl  and  Track  Sup 
(e.g.  fixed  reference  points,  area  of  patrol  etc). 

■  Brief  watch  handover 

■  First  trial  run  with  scenario  with  interruptions  to  assess  data  analysis  methodology 

■  Second  trial  run 

■  Third  trial  run 

■  Debrief  participants 

■  Review  trial  playback 

■  Analyse  outcome 

Day  3: 

■  Brief  new  Navy  SME  participants 

■  Create  necessary  scenario  overlays  by  RTl  and  Track  Sup 
(e.g.  fixed  reference  points,  area  of  patrol  etc). 

■  Conduct  watch  handover  briefing 

■  First  trial  run  with  ASCACT  air  profile  patterns 

■  Break 

■  Second  trial  run  with  ASCACT  air  profile  patterns 

■  Debrief  team/washup 

■  Analyse/review  trial  runs 
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4.  Results^ 


4.1  Scenario 

4.1.1  Scenario  development 

The  scenario  was  created  from  scratch  as  there  were  no  suitable,  pre-existing  NCOT  scenarios 
available.  The  scenario  was  based  on  a  CSTC  training  scenario  (OP  CRATER)  and  was 
prepared  using  the  existing  game  entities  that  were  in  the  NCOT  database,  modified,  where 
necessary,  for  present  requirements.  The  scenario  was  first  planned  on  paper  with  an  outline  of 
the  geographical,  political  and  military  contexts,  to  which  was  added  a  Master  Scenario  Events 
List  (MSEL).  The  latter  provided  a  time  ordered  description  of  all  of  the  game  entities  that 
would  be  introduced  during  the  scenario  and  their  associated  trajectories.  The  full  scenario 
description,  briefing  notes,  order  of  battle,  ID  criteria,  ROE  and  MSEL  are  provided  in  Annex 
A.  The  scenario  was  created  by  HSI*  staff  (ex  ORO  and  Sea  Trainer)  with  assistance  from  the 
NCOT  manager.  The  scenario  was  intended  to  play  for  approximately  2.5  hours.  The  resulting 
master  scenario  file  was  stored  on  the  NCOT  system  for  later  playback  during  the  POC  trial. 

4.1.2  Did  the  scenario  unfold  in  real-time  in  the  manner  intended? 

In  general,  the  playback  of  the  scenario  in  real  time  was  found  to  be  satisfactory.  Entities  moved 
in  the  manner  planned  and  appeared  to  be  realistic  to  the  Ops  Room  team.  Some  dynamic 
entities  became  de-activated  unexpectedly  during  the  course  of  the  playback  but  this  did  not 
appear  to  have  any  apparent  consequences  to  the  players.  The  reason  for  these  de-activations 
was  unclear.  They  could  have  been  caused  by  operator  error,  but  the  nature  of  the  de-activations 
suggests  that  they  may  have  been  caused  by  an  error  in  the  NCOT  software.  Some  entities  did 
not  have  appropriate  characteristics,  for  example  Mirage  aircraft  were  inappropriately  provided 
with  IFF.  However,  such  problems  can  be  readily  rectified  by  correcting  the  appropriate 
database  entry. 

4.1.3  Were  the  mission  foreground  threat  events  suitable? 

Air  and  surface  threat  events  moved  in  the  scenario  appropriately  and  appeared  to  produce  the 
appropriate  responses  among  the  team.  That  is,  they  were  distinguished  from  the  background 
events  and  initiated  the  expected  level  of  assessment  and  analysis. 

4.1.4  Was  the  scenario  suitable  for  producing  the  required  level  of  workload? 

The  team  seemed  to  be  operating  at  a  moderate  level  of  workload.  They  were  able  to  keep  up 
with  the  rate  of  events  and  did  not  appear  to  be  dropping  tasks.  It  should  be  noted  that  there 
were  no  overlapping  air  and  surface  threat  events.  In  order  to  increase  the  workload  level,  the 


‘  In  the  subsequent  text  recommendations  are  highlighted  with  italics. 
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pace  of  air  threats  could  be  increased  and  overlapping  temporal  threats  could  be  implemented. 
Further,  there  is  a  need  to  create  some  realistic  level  of  background  tasks  that  the  ORO  would 
normally  face,  such  as  managing  the  ship’s  Flex  program  or  helo  flying  program  and  dealing 
with  communications.  This  will  ensure  that  the  ORO  will  need  to  divide  attention  over  the 
normal  task  range  for  an  ORO  rather  than  to  able  to  focus  only  on  selected  aspects  of  the  tactical 
picture. 

4.2  General  logistics  for  running  the  scenario 

4.2.1  Time  required  for  set-up  and  preparation 

This  can  be  broken  down  into  the  following  components  and  excludes  scenario  preparation, 
which  was  covered  earlier.  Note  that  the  first  and  second  two  items  can  be  done  in  parallel 

Initial  training  day  for  T&E  SME  actors 

Although  not  required  for  the  POC  trial,  for  all  subsequent  trials  there  will  be  a  need  to  set  aside 
time  for  training  the  Navy  SME's  who  will  serve  as  actors/confederates  while  the  scenario  is 
played  out.  They  will  be  given  an  introduction  to  the  scenario  and  their  roles,  and  will 
participate  in  a  full  rehearsal  of  the  scenario  at  least  once,  and  possibly  twice.  During  the 
rehearsal  they  will  be  trained  to  follow  certain  scripted  requirements  demanded  by  the  T&E  trial 
plan.  Suitable  breaks  were  inserted  at  approximately  2  hour  intervals. 

Initial  physical  setup  -  30  minutes 

This  involves  configuring  the  hardware,  installing  microphones  and  video  recorder. 

Daily  software  setup  -  30  minutes 

This  was  performed  by  the  NCOT  manager  but,  potentially,  could  be  done  by  any  suitable 
trained  staff  member.  The  above  time  allows  for  some  re-starts  as  some  problems  were  observed 
in  some  workstations  that  failed  to  be  initiated  or  went  down. 

Daily  scenario  setup  -  30-60  minutes 

This  involved  having  two  navy  SMEs  (during  the  POC  these  were  the  RTl  and  Track  Sup 
players)  manually  enter  fixed  geographical  entities  into  the  CCS.  This  procedure  had  to  be 
repeated  every  time  the  system  went  down.  This  step  could  be  eliminated  if  a  provision  were 
available  to  create  scenario  overlays  that  could  be  brought  up  at  each  SSD  station.  If  this  cannot 
be  achieved,  then  two  people  would  be  required  to  do  this.  In  this  study,  the  Navy  SMEs  acting 
as  RTl  and  Track  Sup  would  be  available  to  do  this  -  but  HSI*  staff  could  also  be  used. 

Briefing  participants  -  30  minutes 

This  includes,  in  this  order,  a  study  briefing(l5  minutes),  mission  briefing  (15  minutes)  to 
familiarise  study  ORO  subjects  with  study  data  capture  procedures,  and  to  commence  building 
their  “picture”  or  mental  model  of  the  mission  and  operational  context  to  the  point  at  which  it 
would  have  been  at  the  end  of  the  watch  preceding  the  start  of  the  “study”  watch,  discussion  of 
ROE  and  ORBAT  issues  with  ORO  subjects. 
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Mini-watch-30  minutes.  This  provides  an  opportunity  for  participants  to  become  familiar  with 
the  specific  hardware  and  to  start  to  build  the  picture  under  light  background  load  conditions. 
After  the  mission  briefing,  participants  (and  actors)  go  through  a  mini-watch  (say  20  minutes), 
which  would  comprise  the  following  elements.  Starting  a  watch;  doing  a  comms  check;  some 
light  picture  building  (no  surprises  -  and  told  that  before  they  start  this  mini-watch);  settling  in, 
becoming  familiar  with  the  “picture”;  building  their  mental  model  about  what  is  going  on; 
learning  the  sound  of  the  voices  of  the  different  team  members,  how  they  request  and  receive 
information  from  sources  not  physically  present  in  the  scenario;  getting  used  to  the  little  NCOT 
peculiarities  and  room  layout,  also  any  study  specials  (text  message  handouts,  stateboard 
substitutes,  pauses,  SitReps,  etc).  In  general  the  goal  is  to  achieve  the  necessary  familiarity  so 
that  they  would  not  be  experiencing  these  for  the  first  time  in  the  study  watch  proper.  Also, 
during  this  period  they  get  to  ask  any  questions  they  want  about  the  way  things  are  going  to 
work. 

As  a  result,  when  they  start  the  study  watch,  it  will  be  more  as  if  they  are  coming  back  on  watch 
to  something  familiar,  but  just  shifted  forward  in  time. 

Watch  Handover  (10  minutes)  :  Standard  mission  briefing.  ORO  provides  SitRep  at  end  to 
demonstrate  understanding 

4.2.2  Personnel  support 
NCOT  technical  support 

Support  was  provided  by  the  NCOT  manager  who  is  knowledgeable  in  matters  relating  to  the 
running  of  the  NCOT  software  and  issues  relating  to  hardware.  No  local  software  engineering 
support  is  available  to  deal  with  issues  relating  to  the  Unix  operating  environment.  Such  support 
is  currently  provided  through  telephone  to  Richmond,  B.C.  Some  issues  concerning  software 
capabilities  (such  as  the  size  of  scenario  log  files  and  the  over-writing  of  scenario  log  files)  arose 
that  could  not  be  satisfactorily  answered. 

We  recomnend  that  system  technical  support  be  provided  on  site  during  the  initial  setup  day  for 
future  trials. 

Navy  SMEs:  technical  support  and  role  players 

Technical  support 

Normally,  NCOT  is  supported  by  Navy  training  personnel  familiar  with  functions  of  the 
individual  Ops  Room  team  roles  and  the  NCOT  system.  During  the  POC  trial  none  of  these 
were  available,  thus  if  problems  occurred  with  a  specialized  aspect  of  the  CCS  functionality  there 
was  no  one  at  hand  to  check  out  the  problem  and  provide  a  solution.  An  example  of  such  a 
problem  that  had  impact  on  the  fidelity  of  the  simulation  was  our  inability  to  flash  up  fire  control 
radar  as  expected  on  a  contact  in  order  to  get  altitude  information.  Also,  there  was  uncertainty 
about  the  provision  of  EW  bearing  lines  in  the  absence  of  a  CANEWS  work  station  and  operator. 

We  recommend  that  the  appropriate  NCOT  Navy  training  specialists  be  tnade  available  on  the 
set-up  day  of  fiiture  trials  to  ensure  that  the  NCOT  system  is  providing  the  appropriate  Ops  Room 
functionality  for  all  systems  to  be  implemented  in  the  trial. 
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Role  players 

For  the  POC  trial,  experienced  RTl  and  Track  Sup  operators  from  CFNOS  were  provided  and 
met  the  requirements  for  the  trial.  The  ORO  position  was  filled  by  an  ASWC  who  is  currently 
taking  the  ORO  course  and  the  SWC  position  was  played  by  someone  who  had  not  performed 
that  role  in  10  years  and  only  had  operational  experience  as  a  SWC  with  a  ADLIPS  display,  not  a 
HALIFAX  SSD. 

This  experience  highlights  the  fact  that  future  HSt  requests  for  Navy  personnel  must  be  precise 
and  provide  clear  requirements  for  the  experience  level  required  for  each  team  player  and  for  the 
Navy  to  concur  -  or  put  the  study  at  risk.  Failure  to  obtain  personnel  with  the  appropriate 
experience  will  clearly  have  the  potential  to  render  any  ensuing  data  as  unrepresentative  ol  true 
operational  performance.  It  should  be  acknowledged  that  not  having  individuals  with  the 
experience  requested  for  the  POC  trial  may  have  been  due  to  the  short  4  week  lead  time  available 
to  the  Navy  in  the  request  for  personnel.  We  need  direction  with  respect  to  realistic  lead  times 
required,  and  a  better  understanding  of  likely  sources  of  suitable  navy  personnel. 

T&E  support  staff 

The  following  positions  were  found  to  be  required  to  support  scenario  implementation  and  data 
recording  of  the  trial. 

1.  Trial  Co-ordinator:  watches  closely  the  unfolding  of  the  mission  events  on  the  CCS  and 
provides  real-time  co-ordination  and  adjustment  of  the  scenario  events  to  the  NCOT  Instructor 
Stations  and  data  capture  among  the  three  previous  positions.  Filled  by  HSI*  staff  with  extended 
training.  This  position  is  also  the  overall  trial  co-ordinator  and  is  responsible  for  managing  all 
aspects  of  the  trial  on  the  day. 

2.  Data  driver  1:  This  person  controls  the  background  air  and  surface  tracks  that  must  be 
inserted  or  manipulated  in  some  way  in  real  time.  An  NCOT  Instructor  Station  would  be  used  for 
this  position.  Can  be  filled  by  HSI*  staff  with  minimal  training. 

3.  Data  driver  2:  This  person  controls  all  of  the  air  and  surface  threat  tracks  that  must  be 
inserted  or  manipulated  in  some  way  in  real  time.  An  NCOT  Instructor  Station  would  be  used  for 
this  position.  Can  be  filled  by  HSI*  staff  with  extended  training. 

4.  Information  provider:  This  person  responds  to  all  communications  from  the  team  with 
respect  to  requests  for  information  and  simulates  the  following  roles:  TG,  CO,  CANEWS,  ORS 
(and  in  the  future  ASWC  and  TAS  Sup).  An  NCOT  SSD  station  would  be  used  for  this  position. 
Can  be  filled  by  HSI*  staff  with  Navy  Ops  Room  experience  (e.g.  ex  navy  ORO). 

5.  ORO  observer:  sits  behind  the  ORO  and  observes/assesses  ORO  actions  in  real  time. 

Requires  SME  with  ORO  training  experience  -  be  provided  by  HSI*  team. 

Comments. 

Initially  in  the  POC  trial  we  tried  to  combine  positions  3  (data  driver)  and  4  (information 
provider)  into  a  single  role.  However,  the  workload  proved  to  be  too  high  for  a  single  individual 
such  that  communications  to  and  from  the  team  were  subject  to  operationally  unrealistic  delays  or 
unintentionally  inaccurate  responses.  Much  of  this  workload  resulted  from  EWS  traffic  - 
particularly  related  to  the  absence  of  automatic  EW  bearing  lines  from  CANEWS. 
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Positions  1  and  2  are  only  required  because  of  limitations  in  the  existing  NCOT  software  that 
currently  require  entities  with  changes  in  their  trajectories  to  be  entered  into  the  scenario 
manually.  Ideally,  we  would  like  to  pre-script  all  events  and  have  all  entities  enter  into  the 
scenario  and  follow  their  scripted  trajectories  automatically.  Such  a  script  is  simple  for 
background  air  and  surface  contacts  that  move  more  or  less  on  a  single  trajectory  across  the  area 
of  interest.  In  the  case  of  threat  and  other  similar  entities,  whose  trajectories  may  involve 
frequent  changes  in  course,  altitude  or  speed,  the  script  could  include  all  of  these  desired 
characteristics,  such  that  the  entities  always  move  in  a  known  prescribed  pattern.  Therefore,  one 
major  software  improvement  that  would  obviously  reduce  T&E  personnel  support  would  be  to 
provide  additional  software  functionality  that  would  allow  entities  to  automatically  enter  the 
scenario  and  have  their  trajectories  controlled  by  the  NCOT  software  following  a  time-ordered 
script.  This  requirement  corresponds  to  the  existing  NCOT  function  of  "Predetermined 
Tracks"  which  allows  the  creation  of  pre-determined  waypoints  for  an  entity  to  follow.  The 
preparation  work  for  the  scenario  revealed  that  this  function  was  not  operating  in  the  expected 
manner.  This  is  an  important  NCOT  function  for  the  support  of  the  T&E  program  since  it  allows 
the  creation  of  tracks  that  can  also  follow  a  prescribed  path  such  as  zigzag,  ellipse  and  sinuation 
as  well  as  formation  flying  in  which  cohort  entities  automatically  follow  the  pattern  of  the 
formation  leader.  The  actual  availability  of  this  function  needs  to  be  checked  as  soon  as  possible. 

4.2.3  Robustness  &  reliability  of  system  hardware  &  software  during  piay 

The  system  malfunctioned  several  times  during  the  course  of  the  POC  trial.  The  most  frequent 
type  of  failure  was  a  freezing  of  the  scenario  and  workstation,  which  then  required  a  re-boot  of 
the  entire  system  (i.e.  all  work  stations).  The  malfunctions  appeared  to  be  mostly  software 
induced,  although  in  one  case  there  was  a  hardware  failure.  Further,  each  workstation  had  a 
"sleep"  function,  such  that  in  the  absence  of  any  keyboard  or  mouse  activity  for  a  period  of  time 
(the  exact  period  was  not  known  and  differed  for  each  workstation)  the  workstation  would  shut 
down  and  the  entire  network  would  require  re-booting.  This  resulted  in  a  time-out  of 
approximately  10-15  minutes  while  the  scenario  was  restarted.  A  further  10-20  minutes  was  also 
required  to  allow  the  RTl  and  Track  Sup  to  re-enter  CCS  data  such  as  geographical  points  etc. 
Yet  more  time  was  required,  since  the  scenario  could  not  fast  forward  to  the  point  of  stoppage 
without  the  re-creation  in  real  time  of  the  threat  and  background  tracks.  This  means  having  to 
replay  parts  of  the  scenario  that  the  team  will  already  have  been  through.  It  is  recommended  that 
the  "sleep"  function  either  be  switched  off  or  adjusted  to  a  much  longer  time-out  interval. 

A  possible  work-around  for  this  problem  would  be  to  save  the  scenario  file  in  sequential 
fragments  of  30  minute  duration.  Using  the  appropriate  file  segment  following  a  stoppage  would 
mean  a  worst  case  situation  where  15  minutes  of  scenario  would  be  repeated.  The  downside  of 
this  would  be  the  time  required  to  rebuild  the  scenario  overlays  and  a  loss  of  operational  realism. 

A  second  type  of  failure  appeared  to  be  related  to  the  specific  functionality  of  some  of  the 
simulated  Halifax  class  systems.  Such  failures  were  noted  as  follows: 

■  SG150  radar:  failed  to  auto-initiate  and  auto-track  surface  tracks,  even  when 

manually  generated  tracks  were  assigned  to  the  radar 
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•  STIR  fire  control  radars:  failed  to  respond  to  primary  designation  by  SWC,  thereby 
not  providing  altitude  and  fire  control  information  on  tlie  contact 

A  third,  and  possibly  less  important  problem,  was  found  with  the  NCOT  landmass  database  in 
the  area  of  operations.  Small  islands  north  of  the  patrol  area  were  not  depicted  and  there  was  a 
westward  exaggeration  of  about  6  miles  of  the  Yemen  coastline.  While  causing  some  reduction 
in  operational  realism,  we  believe  that  this  problem  is  manageable 

Our  preliminary  assessment  is  that  such  failures  give  rise  to  some  concerns  about  whether  the 
NCOT  facility  can  provide  the  required  level  of  reliability  to  support  the  T&E  effort.  A  final 
judgment  of  this  should  probably  be  suspended  until  after  the  pilot  trial  has  been  conducted  since 
some  problems  may  simply  require  better  insight  into  existing  system  functionality. 

4.2.4  Ability  to  record  additional  audio  and  video 

A  video  camera  was  mounted  towards  the  rear  of  the  team  and  appeared  to  adequately  capmre 
the  movements  of  the  team  (e.g.  head  position,  body  movements,  face  to  face  interaction). 
Microphones  were  mounted  between  the  ORO  and  SWC  and  between  the  RTl  and  Track  Sup; 
and  these  appeared  to  be  adequate  for  capturing  off-net  verbal  communications  and  verbal  input 
to  radio  microphones  from  those  positions. 

4.2.5  Suitability  and  adequacy  of  communications  among  scenario  players 

The  NCOT  communication  facility  appeared  to  be  adequate  in  providing  the  requisite  network 
communications  among  the  team,  although  some  of  the  circuits  were  not  able  to  be  assigned  their 
usual  SHINCOMS  reference.  For  example,  the  internal  Above  Water  Warfare  coordination  net 
was  assigned  to  the  ASW  button  instead  of  the  AWW  button.  The  quality  and  speed  of 
communications  appeared  to  adequately  mimic  normal  Ops  Room  circumstances.  Recording 
quality  was  marred  by  noise  from  alarms,  which  blotted  out  some  recordings.  There  was  also  a 
background  hum  from  the  ventilation  system  that  made  softly  spoken  communications  difficult  to 
hear. 

4.2.6  Ability  to  support  specific  T&E  requirements 
4. 2.6. 1  Control  of  game  entities  in  real  time 

Game  entities  are  described  in  detail  in  Annex  A.  Once  introduced  into  the  scenario,  game 
entities  generally  moved  in  an  operationally  realistic  manner.  There  were  three  types  of  entity 
patterns  that  needed  to  be  controlled  -  single  trajectory  tracks,  tracks  that  had  several  legs  and 
multiple  trajectories  and  tracks  that  may  need  to  respond  or  react  in  an  appropriate  manner  in 
real  time  to  actions  taken  by  the  Ops  Room  team. 

Single  trajectory  entities  are  brought  into  the  game  at  a  known  time  and  reference  point  and 
follow  a  ballistic  pattern  across  the  area  of  operations.  Examples  include,  commercial  air  traffic 
taking  off  and  landing  ,  helicopter  traffic  between  the  coast  and  oil-rigs  and  surface  vessels 
sailing  from  ports.  The  T&E  team  controlled  these  entities  by  taking  them  from  a  "parked"  area 
that  was  outside  the  area  of  operations  and  moving  them  to  a  waypoint  and  activating  them.  At 
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this  point  the  scenario  software  then  took  over  their  movement.  This  manual  control  process  was 
required  because  the  current  version  of  software  does  not  support  the  timed  entry  of  entities  into 
the  scenario  following  a  pre-scripted  schedule. 

The  second  type  of  tracks  involved  multiple  trajectories,  that  is  entities  that  might  change  course, 
altitude  or  speed  while  in  the  area  of  operations.  For  example,  Yemeni  aircraft  taking  off  from 
Hays  airbase  could  follow  a  variety  of  patterns  that  might  provide  different  levels  of  threat  and 
interest.  Each  individual  change  of  pattern  required  dynamic  real-time  control  by  a  member  of 
the  T&E  team. 

This  proved  to  be  a  manageable  task,  but  was  subject  to  a  number  of  limitations.  First,  as  the 
number  of  entities  in  a  complex  scenario  that  require  real-time  control  increases,  the  ability  of  the 
entity  controller  to  manage  all  of  these  becomes  compromised.  Second,  there  is  some 
imprecision  in  following  the  required  trajectory  script  exactly  and  to  repeat  that  pattern  in  the 
same  manner  on  successive  trials.  This  results  from  small  lags  in  initiating  events  and  latencies 
in  aircraft  flight  dynamics,  as  well  as  the  skill  level  of  the  operator.  Approximate  patterns  can  be 
repeated  with  ease;  however,  this  will  result  in  some  imprecision  in  collecting  MOP  response 
time  data  where  the  start  time  is  initiated  by  a  track  that  is  supposed  to  be  at  an  exact 
geographical  location,  relative  to  the  ship.  We  were  not  able  to  establish  the  magnitude  of  such 
potential  errors  because  of  problems  with  scenario  playback  (see  below). 

The  requirement  for  precise  repeatability  of  tracks  is  particularly  relevant  to  the  generation  of 
tracks  that  are  designed  to  produce  some  radar  confusion,  as  exemplified  by  the  patterns  used  in 
ASCACT  trial.  Three  of  the  ASCAT  patterns  were  evaluated.  Pattern  T.  two  aircraft  at  same 
altitude  and  speed,  on  converging  tracks,  converge  and  fly  close  together  directly  over  the  AOI 
before  diverging.  Pattern  2:  two  aircraft  at  the  same  altitude  and  speed  have  trajectories  that 
cross  backwards  and  forwards  with  each  other  across  the  AOI.  Pattern  3:  two  A/C  on  same 
heading  and  speed  approach  AOI,  one  A/C  changes  altitude  lower,  and  maintains  lower  altitude 
while  holding  same  direction  as  other  A/C. 

In  general,  it  was  found  difficult  to  generate  tracks  involving  two  A/C  doing  such  close 
manoeuvring  with  exact  repeatable  precision,  although,  with  practice,  a  reasonable 
approximation  to  the  required  patterns  could  be  produced.  Further,  some  difficulty  was 
encountered  in  getting  entities  to  make  fast  turns  in  the  manner  that  accurately  simulates  combat 
manoeuvres. 

One  important  observation  was  that  the  NCOT  simulated  radar  seemed  to  experience  no  difficulty 
in  maintaining  separate  and  correct  track  identities  for  the  ASCACT  tracks  that  we  were  able  to 
reproduce,  such  as  two  A/C  coming  together  from  different  vectors.  Therefore,  it  remains  a 
moot  point  whether  to  continue  to  use  such  tracks  to  simulate  existing  radar  tracking  problems 
that  MSDF  technology  is  designed  to  solve.  Therefore,  we  recommend  an  early  discussion  with 
DREA  and  LM  to  further  elaborate  upon  the  development  of  track  requirements  that  will  suitably 
address  existing  radar  technology  problems  and  MSDF  improvements.  We  further  recommend 
that  DCIEM  put  in  place  the  necessary  contractual  arrangement  to  allow  HSI*  to  interact  and 
work  closely  with  LM  to  ensure  rapid  and  timely  interchange  of  technical  information  concerning 
the  progress  of  COMDAT  support  work. 
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The  third  type  of  track,  which  requires  the  game  entity  to  move  in  an  operationally  realistic 
manner  in  response  to  the  appropriate  circumstances  in  real  time,  would  require  real-time  manual 
control  by  the  T&E  team  in  any  case. 

Our  assessment  is  that  in  order  to  reduce  T&E  overhead  for  manual  control  of  entities  and  to 
ensure  precision  in  the  execution  of  track  trajectories  the  software  be  enhanced  to  allow  pre¬ 
scripted  tracks  to  enter  and  move  within  the  scenario  under  game  software  control.  ^ 

4.2.6. 2  Communications  from  T&E/actors  to  Ops  Room  Team 

One  member  of  the  T&E  team  played  multiple  roles  to  provide  contextual  communication  and 
information  to  the  Ops  Room  team  by  way  of  one  of  the  radio  nets  in  response  to  requests  for 
information  from  the  CO,  OOW,  CANEWS  etc.  This  appeared  to  work  effectively  and  in  a 
timely  manner.  The  exception,  noted  earlier,  was  when  this  same  member  of  the  T&E  team  was 
concentrating  on  the  control  of  game  entities,  which  resulted  in  a  lag  in  the  communication 
response. 

In  future  trials,  it  would  appear  possible  and  desirable  to  pass  paper  text  messages  to  the  ORO 
using  the  ORO  observer  position.  These  would  make  the  ORO  task  more  realistic,  and  include  a 
requirement  to  select  and  forward  relevant  information  to  others  in  the  Ops  room  team. 

4.2.6.3  Consistency  of  scenario  repetition  across  trials 

Although  we  were  not  able  to  assess  this  formally  because  of  playback  problems,  we  were  able  to 
conclude  that  background  air  and  surface  events  could  be  repeated  with  the  required  precision 
across  trials.  However,  there  were  problems  in  maintaining  the  required  consistency  of  both  the 
timing  and  spatial  precision  of  dynamically  controlled  tracks,  as  outlined  in  4.2.6. 1. 

4. 2. 6. 4  Support  for  collecting  specific  MOPs 

The  detailed  MOPs  provided  in  reference  1  were  reviewed  and  a  generic  list  of  requirements  to 
support  the  delivery  or  required  scenario  events  and  data  collection  was  constructed.  The  generic 
capability  was  derived  by  examining  all  of  the  individual  scenario  input,  monitoring  and 
recording  elements,  and  then  grouping  them  into  more  generic  categories.  This  information  is 
shown  in  the  Tables  1-5  below,  together  with  an  indication  of  the  ability  to  support  these 
requirements  within  NCOT  and  by  the  T&E  team. 

Table  1  provides  a  summary  assessment  from  the  POC  of  the  capability  of  NCOT  to  provide 
information  that  must  be  provided  to  the  ORO  other  than  through  the  specific  scenario  events. 
This  information  could  be  text  messages  by  hand  or  via  the  CCS,  or  voice  messages  through  the 
various  nets  or  face  to  face.  The  meaning  of  the  various  columns  is  as  follows: 

Info  to  ORO:  describes  the  type  of  message  and  medium 

Feasible:  indicates  our  assessment  of  whether  it  can  be  done  in  NCOT 


^  We  have  been  told  that  this  was  supposed  to  be  achievable  through  the  "Pre-Determined  Track”  lunction  ol  the  Scenario  Builder 
software,  however,  we  have  not  had  any  success  in  making  this  work  correctly 
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Timing  sent:  means  if  the  time  of  the  message  transmission  can  be  recorded  and  how  this  is 
accomplished. 

Timing  reed:  means  if  the  time  of  the  message  reception  by  the  ORO  can  be  recorded  and  how 
this  is  accomplished. 

T&E-who  does:  indicates  who  in  the  T&E  team  would  be  responsible  for  delivering  the  message. 

If  not  T&E,  ID  source:  If  the  message  does  not  originate  with  the  T&E  team,  can  the  message 
source  be  identified  and  how. 

Capture  Response  Action:  If  the  message  results  in  an  ORO  action  (response)  can  this  event  be 
captured  and  how. 

Capture  Response  Content:  If  the  message  results  in  an  ORO  action  (response)  can  the  content 
be  captured  and  how. 

ID  target  of  response:  If  the  message  results  in  a  response  by  the  ORO  can  the  target  (recipient) 
of  the  response  be  identified. 

Time  of  response:  If  the  message  results  in  a  response  by  the  ORO  can  the  time  of  the  response 
be  captured. 

Assess  response:  If  the  message  results  in  a  response  by  the  ORO  can  the  content  of  the  response 
be  analysed  and  how. 

In  Table  2  ,  we  assess  the  outcome  of  the  POC  in  terms  of  the  capabilities  of  NCOT  to  provide 
support  for  the  following  input  events 

Background  air/ surface  commercial 
Background  air/surface  hostile 

Significant  A/C  actions  (changes  course  to  TG,  altitude,  speed,  radar,  previous 
pattern,  assumes  attack  profile 

ASCACT  aircraft  patterns  thought  to  generate  radar  confusion. 

For  each  of  these  we  determined  that  each  of  the  following  required  pieces  of  data  for 
measurement  purposes  could  be  obtained  from  the  scenario  or  playback: 

time  into  scenario 

time  painted  on  radar  display 

time  into  area  of  interest 

time  out  of  area  of  interest 

time  of  change  ins  status  of  contact 

time  at  own  ship's  weapons  range 

time  at  A/C  weapons  range 

time  EWS  emitted 

time  EWS  detected. 

While  the  majority  of  these  input  events  can  be  easily  implemented  in  NCOT  and  the  required 
data  captured,  some  technical  difficulties  in  simulating  ASCACT  tracks  to  the  required  degree  of 
precision  was  encountered  (as  mentioned  earlier  in  section  4.2.6. 1). 


Humaayyi'tew^  Incorporated® 


NCOT:  Proof  of  Concept 


Page  19 


P517839.PDF  [Page:  22  of  39] 


In  Table  3  we  show  how  in  NCOT  we  can  capture  data  relevant  to  ORO  actions  on  the  CCS,  and 
interactions  with  other  information  sources.  Tables  4  and  5  show  how  we  can  record  ORO  and 
other  communications  and  assess  ORO  knowledge,  respectively.  This  capability  is  assessed  in 
terms  of  how  we  would  specifically  capture  information  content,  the  time  of  the  action  and  the 
identity  of  the  recipient  of  ORO  communications.  The  ability  to  add  T&E  embedded  probes  into 
the  scenario  is  also  assessed  using  approaches  such  as  a  SITREP  to  the  CO,  a  "bogus"  request 
for  information  as  well  as  the  means  for  determining  the  associated  response  time  to  such 
embedded  probes. 

In  general,  NCOT  appears  to  be  capable  of  supporting  the  range  of  techniques  that  will  be 
necessary  to  capture  MOPs.  Having  said  that  however,  it  should  be  noted  that  at  this  time  we 
have  not  had  the  opportunity  to  actually  calculate  sample  MOPs  from  the  POC  data  reci  rd.  This 
arose  because  of  a  number  of  problems  relating  to  the  lack  of  reliability  in  the  system  functioning 
and  especially  in  playback  (see  below).  Clearly,  the  pilot  trial  will  provide  the  most  opportune 
time  for  assessing  the  feasibility  of  capturing  and  analysing  the  data  in  the  manner  intended. 

4.2.6.5  Recording  of  trial  data 

For  each  workstation  the  screen  content  and  all  net  based  communications  are  continuously 
recorded  on  the  hard  drive  of  the  individual  workstation.  This  information  is  stored  in  a 
proprietary  file  format  and  is  not  amenable  to  being  read  using  standard  data  analysis  or 
statistical  software,  .  The  entire  scenario  session  may  be  recorded  as  a  single  file,  or  recording 
can  be  started  and  stopped  under  real-time  control  to  yield  smaller,  sequential  data  files.  This 
may  have  some  advantages  given  limitations  in  data  playback  outlined  below. 

However,  file  management  functionality  appeared  rudimentary,  and  could  be  improved.  For 
instance,  we  were  not  able  to  access  information  concerning  the  length  of  the  resulting  file. 
Whether  this  is  a  result  of  a  software  limitation  or  lack  of  knowledge  on  our  part  on  how  to 
retrieve  such  information  is  not  known.  We  also  learned  that  once  a  hard  drive  becomes  full  the 
system  starts  to  overwrite  existing  files;  again  how  this  actually  happens  and  the  consequences 
for  stored  data  from  trials  is  unknown.  At  present  files  cannot  be  copied  or  removed  from  the 
hard  drive  onto  secondary  mass  storage  media. 

These  issues  raise  some  concerns  for  the  future  management  and  access  of  T&E  data.  Hence,  we 
recommend  that  a  capability  be  added  to  store  data  files  on  an  external  medium.  This  may 
require  a  DVD  format  given  that  the  length  of  the  file  may  exceed  CD-ROM  capacity.  A  further 
advantage  of  storage  on  such  a  medium  is  outlined  in  the  following  section  concerning  playback 
of  stored  data.  Whatever  medium  is  chosen,  this  requirement  is  considered  essential  for  the 
retention  of  critical  trial  data  for  the  production  of  which  has  been  the  result  of  considerable 
logistical  effort,  which,  obviously,  cannot  afford  to  be  wasted. 
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4.2.6.6  Playback  and  analysis  of  NCOT  data  files 
Playback 

Some  serious  limitations  to  the  playback  of  NCOT  data  were  discovered  and  are  outlined  below. 

1.  Data  files  are  currently  'owned'  and  located  on  each  individual  workstation,  as  a 
consequence,  they  can  only  be  played  back  on  that  station.  The  playback  may  be  monitored  on  a 
separate  workstation  that  is  temporarily  configured  for  this  purpose  on  the  NCOT  network.  The 
NCOT  system  does  not  produce  a  single  data  record  integrated  across  all  of  the  workstations  that 
comprise  a  small  team  configuration  for  a  particular  trial.  This  lack  of  capability  may  have 
significant  consequences  for  trial  analysis.  Using  the  NCOT  playback,  the  simultaneous  review 
and  analysis  of  the  T&E  trial  data  across  all  team  workstations  would  require  a  co-ordinated 
manual  start  of  the  replay  on  each  of  the  individual  workstations  and  an  observer/analyst  at  each 
workstation.  While,  with  some  minimal  practice,  any  error  in  ensuring  a  common  start  of  the 
replay  would  probably  be  acceptable,  we  have  not  been  able  to  ascertain  whether  the  playback  at 
each  workstation  will  remain  synchronised  over  what  could  be  the  course  of  a  2  hour  trial. 

2.  Data  files  can  only  be  played  back  in  real  time.  Further,  there  are  no  capabilities  to  fast 
forward,  slow  forward,  reverse,  pause  or  stop/restart.  In  fact,  once  the  replay  has  been  stopped 
when  it  is  re-started  the  playback  reverts  to  the  beginning  of  the  file!  Clearly,  such  limitations 
severely  compromise  the  ability  of  the  T&E  team  to  perform  the  required  fine  grain  analysis  of 
the  trial  data  in  anything  approaching  an  efficient  manner,  if  at  all.  What  is  already  likely  to  be  a 
labour  intensive  activity  is  only  made  worse  by  this  apparent  lack  of  a  rudimentary  capability. 

Two  solutions  to  this  problem  are  suggested.  First,  that  the  necessary  software  could  be 
developed  to  provide  the  functionality  to  support  analysis  of  trial  data  (outlined  in  detail  later). 
This  approach  is  likely  to  be  time  consuming  and  costly.  The  second,  and  our  preferred,  solution 
is  to  provide  a  capability  to  capture  the  video  and  audio  output  of  the  workstation  playback  and 
convert  into  a  format  that  will  allow  recording  on  standard  digital  video  media.  The  added 
advantage  of  such  an  approach  is  that  trial  data  could  then  be  taken  off-site  for  analysis  at  HSI* 
premises,  with  significant  savings  in  travel  overhead,  or  the  need  for  the  purchase  of  an  NCOT 
compatible  workstation  for  data  playback.  Such  capture  could  be  accomplished  through  the 
purchase  of  a  high  quality  video  converter  that  would  transform  the  native  NCOT  RGB  video 
output  into  an  NTSC  video  compatible  format.  The  technical  requirements  and  cost  of  such  a 
solution  are  currently  being  investigated. 

3.  Several  system  freezes  were  encountered  during  the  playback  of  recorded  data  from  the 
DCIEM  POC  sessions.^  This  may  have  been  the  result  of  trying  to  playback  several  files 
simultaneously  on  different  workstations  in  order  to  accommodate  our  needs.  On  the  final 
afternoon,  the  system  crashed  during  playback  and  could  not  be  re-started  to  allow  further 
playback.  Telephone  enquiries  to  MDA  headquarters  in  Richmond  did  not  result  in  an  immediate 
solution  being  found  to  the  problem  of  playback  freezing. 


'  This  did  not  happen  as  much  with  the  earlier  recorded  DREA  POC  data  that  was  stored  as  a  single  file  The  DCIEM  data  were  stored 
in  a  sequential  senes  of  smaller  files  in  order  to  avoid  having  to  go  back  to  the  very  start  of  the  scenano,  each  lime  the  playback  was 
stopped 
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4.  Playback  of  audio  traffic  captured  from  the  comnr  nets  was  somewhat  degraded  by  the 
intrusion  of  noise  from  background  fans  of  the  heating/ventilation  system.  Further, 
communications  became  almost  unintelligible  when  a  CCS  alarm  had  been  captured  on  the  audio 
track.  Our  tentative  conclusion  is  that  the  audio  playback  is  of  marginal  quality  for  T&E 
analysis.  Further  investigation  of  this  is  required  to  deterin  ne  whether  there  are  ways  to 
improve  the  signal  to  noise  ratio. 

On  the  positive  side,  the  quality  of  the  video  playback,  whether  on  a  workstation  monitor,  or 
displayed  on  a  large,  front-surface  projection  screen  through  a  high-resolution  video  projector, 
was  sharp  and  provided  clear  detail  of  all  symbology,  alpha-numerics  and  QAB  strokes. 
However,  a  separate  video  record,  that  was  captured  from  a  workstation  monitor  or  the  projected 
image  by  the  T&E  team  digital  video  camera,  showed  poor  image  quality.  Thus,  it  was 
concluded  that  this  would  not  be  a  satisfactory  workaround  to  the  problem  of  creating  a  video 
record  that  would  be  more  amenable  for  T&E  analysis  than  the  NCOT  data  file. 

Analysis 

Notwithstanding  the  above  comments  concerning  limitations,  the  data  files  do  provide  the 
necessary  information  to  allow  detailed  analysis  to  allow  the  extraction  and  calculation  of 
performance  data.  The  workstation  clock  is  visible  at  all  times  and  would  allow  timing  of  events 
(accurate  to  the  nearest  second)  to  be  extracted.  The  effect  of  the  operator's  actions  are  clearly 
seen  on  the  screen  in  terms  of  the  information  provided  in  the  radar  area  as  well  as  in  the  various 
numerical  and  tabular  data  areas.  We  were  able  to  see  which  track  was  currently  hooked,  the 
track  data,  the  range  selected  and  all  other  information  that  provides  the  tactical  picture.  Thus,  it 
would  appear  feasible  to  determine  from  the  video  and  comms  records  the  ongoing  tactical  focus 
of  the  ORO  and  also  allow  some  response  time  measures  to  be  collected,  such  as  the  time 
between  the  appearance  of  new  information  and  the  subsequent  ORO  response.  It  should  be 
noted  that  because  of  the  lack  of  access  to  the  playback  files  in  the  final  session,  we  were  not 
able  to  actually  attempt  to  extract  MOP  data  examples  as  had  been  planned 
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Info  to  ORO 
(other  than 
T&E  software) 

Feasible 

Timing  Sent 

Timing  Reed 

T&E- 

who  does? 

If  not  T&E,  ID 
source 

Capture 

Response 

Action 

Capture 

Response 

Content 

ID  target  of 
response 

Time  of 
response 

Assess 

Response 

Text  message 
via  CCS 

NO 

Text  message 
by  hand 

YES.  via  ORO 
observer. 

YES  ORO  obs 
notes 

WS  clock 

YES  Trial 
coord  notes 
appropriate  WS 
clock 

ORO  obs 

Trial  coord 

Known  to  team 

YES-  NCOT 
video/audio 
record  of  ORO 
WS  or  ORO 
obs. 

YES-  NCOT 
video/audio 
record  of  ORO 
WS  or  ORO 
obs 

YES.  NCOT 
video/audio 
record  of  ORO 
WS  or  ORO 
obs. 

YES  from 
playback  clock 
or  observer 
record. 

Post  event 
analysis  by 

SME 

Voice  message 
from  net 

YES 

YES.  video/ 
audio  playback 

YES.  playback 
clock 

NA 

YES 

video/audio 

playback 

YES-  NCOT 
video/audio 
record  of  ORO 
WSorORO 
obs 

YES-  NCOT 
video/audio 
record  of  ORO 
WS  or  ORO 
obs 

YES.  NCOT 
video/audio 
record  of  ORO 
WSorORO 
obs. 

YES.  from 
playback  clock 
or  observer 
record 

Post  event 
analysis  by 

SME 

Voice  message 
face  to  face 

YES.  from  T&E 
video/ 

audio  record 

YES  need  to 
add  clock  to 
T&E  video 
record 

YES- 

NA 

YES-  from  T&E 

video/audio 

record 

YES-  NCOT 
video/audio 
record  of  ORO 
WSorORO 
obs. 

YES.  NCOT 
video/audio 
record  of  ORO 
WS  or  ORO 
obs 

YES.  NCOT 
video/audio 
record  of  ORO 
WS  or  ORO 
obs. 

YES:  from 
playback  clock 
or  observer 
record. 

Post  event 
analysis  by 

SME 

Table  1:  Feasibility  and  methods  for  providing  information  to  ORO  and  for  capturing  MOPs 
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Input  events 
from  scenario 

Time  into 
scenario 

Time 

painted 

Time  into  AOI 

Time 

symbology 

Time 

Out  of  AOI 

Time  change 
in  status 

Time  at  own 
ship  weapons 
range 

Time  at  A/C 
weapons 
range 

Time  EWS 
emitted 

Time  EWS 
detected 

Background 

air/surtace- 

commercial 

Background 

surface 

Hostile  air 

Hostile  surface 
A/C  changes 
course  to  TG 

A/C  changes 
speed 

/VC  changes  alt 
A/C  radar 

A/C  changes 
previous 
pattern 
/VC  assumes 
attack  profile 
Neutral- 
changes  profile 
ASCACT 
patterns 

YES.  from 
ground  truth 
playback  of 
scenario 

YES:  from 
ground  truth 
playback  of 
scenario 

YES-  from 
ground  truth 
playback  of 
scenario 

YES:  from 
playback  of  trial 
scenario 

YES,  from 
ground  truth 
playback  of 
scenario 

YES.  from 
ground  truth 
playback  of 
scenario 

YES'  from 
ground  truth 
playback  of 
scenario 

YES.  from 
ground  truth 
playback  of 
scenario 

YES'  from 
ground  truth 
playback  of 
scenario 

YES.  from 
ground  truth 
playback  of 
scenario 

Table  2:  Feasibility  and  method  of  implementing  and  recording  scenario  events 
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ORO  actions 

Capture  screen 

Capture/ timestamp  action 

Assess  action 

ORO  action  on  CCS 

YES  NCOT  playback 

YES  NCOT  playback  Accurate  to 
nearest  second. 

ORO  obs  in  real  time  commentary 
through  headset.  Post  trial  analysis  by 
SME  on  NCOT/T&E  playback 

ORO  consults  other  sources 

ORO  use  of  ancillary  information  (e  g  manuals,  plot,  simulated  stateboards,  tacpacs)  captured  by  T&E  video  and  analysed 
post  event.  May  also  be  captured  by  ORO  obs 

Table  3:  Feasibility  and  methods  for  capturing  ORO  actions 


ORO  Communications 

Capture  message  content 

Capture/  timestamp  action 

ID  recipient 

Assess  content 

ORO  speaks  directly  to 
team/member  (not  on  net) 

YES.  T&E  audio  capture 

YES.  T&E  audio  file  playback 

Will  need  to  add  functionality  to 
provide  visual  time  base 
indicator. 

YES.  T&E  audio  file  playback 

SME  analysis  of  T&E  audio  file 

ORO  uses  net 

YES'  NCOT  audio  capture 

YES'  NCOT  audio  playback 
plus  CCS  clock  (video  record) 

YES'  NCOT  audio  playback 

SME  analysis  of  T&E  audio  file 

Table  4:  Feasibility  and  methods  for  capturing  ORO  communications 
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Assessing  ORO  knowledge 

Real  time  SITREP  to  CO 

ORO  CCS  Display 

T&E  Bogus  request  for  info 

Response  time  to  embedded 
probe 

Picture  content  at  start  of 
scenario 

YES 

YES 

NA 

Change  in  picture  content 

YES 

YES 

NCOT  playback 

Indiv  threat  comprehension 

YES 

YES 

NA 

Change  in  threat  status 

YES 

YES 

YES 

NCOT  playback 

Relative  threat  priorities 

YES 

YES 

YES 

Tactical  situation 

YES 

YES 

Time  to  CPA 

YES 

YES 

YES 

Own  engagement  envelope- 
earliest/latest  point  to  shoot  - 

YES 

Enemy  engagement  range  - 
contact 

YES 

Tabic  5:  Feasibility  and  methods  for  assessing  ORO  knowledge. 
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5.  Conclusions 


The  NCOT  environment  provides  adequate  support  for  the  development  and  running  of 
scenarios  to  support  T&E  testing  and  for  the  collection  of  required  data.  This  assessment  is 
based  upon  the  assumption  that  problems  with  the  fidelity  of  simulation  of  some  system 
functions  (see  4.2.3)  may  be  readily  corrected. 

With  respect  to  the  playback  and  analysis  of  T&E  data  we  believe  that  there  are  some  serious 
limitations  of  the  NCOT  system.  This  conclusion  is  based  upon  the  way  we  expect  that  the  data 
will  have  to  be  analysed  after  the  trial  conclusion.  We  anticipate  that  we  will  need  to  have  a 
high  level  of  control  over  the  scenario  playback  process  in  order  to  extract  the  required  data; 
such  control  will  include  an  ability  to  pause,  rewind,  slow  forward  and  fast  forward.  Further, 
there  may  be  a  need  to  provide  a  time-synchronised  playback  between  two  or  more 
workstations.  In  order  to  achieve  such  control,  significant  software  changes  would  need  to  be 
made  or  alternate  approaches  should  be  considered  (as  outlined  below). 

A  second  concern  relates  to  the  ability  to  manage  archived  scenario  files.  At  present  we  have 
been  advised  that  there  is  no  capability  to  determine  the  length  of  files  or  how  much  space  is 
available  on  the  drive  to  accommodate  new  files  or  to  ensure  that  they  are  not  overwritten. 

There  is  also  no  ability  to  copy  data  files  to  external  devices  at  individual  workstations. 

A  third  concern  is  that,  even  assuming  that  the  above  two  problem  areas  can  be  corrected,  all 
analysis  would  have  to  be  done  at  the  NCOT  facility.  Given  that  one  might  expect  each  hour  of 
recording  to  require  3-4  hours  of  analysis,  this  would  mean  the  T&E  analysts  would  have  to 
spend  several  days  at  NCOT.  This  could  not  only  raise  issues  of  accessibility,  but  also  add 
overhead  costs  to  the  project  associated  with  travel  and  accommodation.  Again,  some  solutions 
to  this  issue  are  outlined  below. 


Humam^.s-rem^’  Incorporated® 


Page  27 


NCOT:  Proof  of  Concept 


P517839.PDF  [Page:  30  of  39] 


6.  Recommendations 


6.1  Software  and  system  improvements 

(in  approximate  order  of  priority) 

Some  of  these  improvements  may  indeed  exist,  but  were  not  apparent  or  could  not  be  made  to 
function  during  the  POC. 

1 .  Provide  a  capability  in  the  master  scenario  file  to  allow  entities  to  follow  a  pre-scripted 
trajectory.  This  could  be  readily  accomplished  by  fixing  the  problems  in  implementing 
the  NCOT  Scenario  Builder  function  of  "Pre-determined  tracks".  (NCOT  Manual 
Section  6-27). 

2.  Allow  entities  to  automatically  enter  into  the  scenario  at  pre-specified  times  and  follow 
a  constant  track  and  speed. 

3.  Allow  entities  to  automatically  enter  into  the  scenario  at  pre-specified  times  and  follow 
a  pre-scripted  trajectory. 

4.  Provide  a  capability  for  a  time-stamped  flag  to  be  entered  from  an  instructor 
workstation  into  the  scenario  record.  This  could  be  achieved  by  capturing  function  key 
presses  and  logging  the  time  of  the  press  and  saving  to  a  file.  The  resulting  data  file 
should  comprise  a  time-ordered  list  of  the  key  presses  and  time  (scenario  time)  when 
pressed.  This  file  should  be  in  an  ASCII  text/tab  delimited  format  to  allow  for  export 
to  a  spreadsheet. 

5 .  Provide  a  capability  to  capture  specific  key  presses  at  the  ORO  and  other  team 
workstations.  These  are  to  be  time  stamped  and  recorded  to  a  data  file  in  the  manner 
described  under  item  4.  Specific  actions  required  to  be  captured  include:  quick  action 
buttons  (QABs),  numeric  keys,  "enter"  key,  F2  and  the  alarm  button. 

6.  Develop  a  capability  for  control  over  playback  files  to  include:  slow  motion,  pause/re- 
start,  rewind,  fast  rewind/fast  forward  and  go  to  flags  (as  in#4). 

7.  Provide  an  ability  to  download  data  files  to  external  media. 

8.  Provide  an  ability  to  replay  multiple  files  simultaneously  on  different  workstations 
(preferably  in  synchrony) 

9.  Provide  data  file  management  capabilities 

10.  Provide  an  override  of  the  workstation  "sleep"  function  or  allow  it  be  adjusted  to  a 
much  longer  time  out. 

1 1 .  On  SSD  enable  dimming  of  symbology  so  that  can  operator  can  examine  radar  paint 
alone  (as  occurs  on  the  actual  CCS) 

12.  Provide  a  capability  for  the  creation  of  CCS  overlays  ahead  of  time  and  to  allow  them 
to  be  integrated  into  the  Tactical  Situation  Area  of  the  SSD  when  the  scenario  is  run. 
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13.  Enable  instructor  to  draw  “lines”  and  save  for  later  use  e.g.  patrol  areas  (e.g.  like 
Word  Draw  options  plus  text  insertions) 

14.  Provide  a  means  of  distinguishing  between  active  tracks  and  stationary  or  parked  tracks 
in  the  instructor  screen  i.e.  to  rapidly  identify  all  tracks  in  play  at  the  moment 

15.  Allow  audio  playback  on  any  workstation 

16.  Improve  signal  to  noise  ratio  (sound  quality)  of  audio  playback. 


6.2  Data  capture  and  analysis 

Notwithstanding  items  3,4  and  5  above,  our  preferred  solution  for  data  analysis  is  to  allow 
data  files  to  be  captured  in  a  manner  that  will  allow  their  removal  from  site  and  analysis  to 
be  conducted  at  an  HSrsite.  To  achieve  this  we  propose  the  following  solutions: 

Capture  the  NCOT  workstation  video  output  to  the  CCS  display  in  real  time  and 
convert  it  to  NTSC  video  using  a  scan  converter.  This  will  be  required  for  each 
workstation  of  interest  (which  for  present  purposes  is  4).  The  subsequent  video  signal 
can  then  be  captured  on  digital  videotape  with  a  single  record  for  each  source. 

Possibly,  if  we  can  maintain  signal  quality,  the  four  records  could  be  multiplexed  onto 
a  single  tape.  For  audio  records,  NCOT  maintains  a  separate  digital  audio  file  for  each 
workstation,  and  these  could  be  subsequently  transferred  to  a  removable  media  to  allow 
integration  with  the  video  record  when  the  analysis  is  performed.  We  are  also 
exploring  a  new  technology  that  will  simultaneously  capture  the  output  of  four 
workstations  and  integrate  them  into  a  single  digital  (non-video)  record  for  subsequent 
payback. 

An  alternative  approach  would  be  to  purchase  and  configure  an  NCOT  compatible 
workstation  for  local  playback/review  purposes.  The  major  disadvantage  of  such  an 
approach  is  that  the  playback  would  be  limited  to  a  single  data  record  at  a  time.  Hence,  the 
co-ordinated  analysis  of  information  across  team  players  at  different  workstations  could  not 
be  achieved. 


6.3  Personnel  requirements  to  support  T&E 

1.  On  the  set-up  day  prior  to  data  capture,  NCOT  Navy  trainers  be  made  available  to 
ensure  that  all  of  the  Ops  Room  simulated  functionality  is  working  in  the  operationally 
correct  manner. 

2.  This  day  will  also  serve  as  a  training  day  for  T&E  SME  actors.  They  will  be  given  an 
introduction  to  the  scenario  and  their  roles,  and  will  participate  in  a  full  rehearsal  of  the 
scenario  at  least  once,  and  possibly  twice.  During  the  rehearsal  they  will  be  trained  to 
follow  certain  scripted  requirements  demanded  by  the  T&E  trial  plan. 

3.  On  the  same  day,  a  systems  software  specialist  should  be  on  hand  to  ensure  that  the 
system  is  functioning  appropriately  and  that  there  is  space  to  store  data  records. 

4.  Navy  provided  SMEs. 

Actors:  1  RTl,  1  Track  Sup,  1  SWC.  All  need  to  have  some  experience,  preferably 
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including  working  together  as  a  single  team.  Thus,  our  ideal  would  be  to  draw  an  intact 
team  from  one  watch  on  one  ship  to  stay  over  both  days.  Recognizing  that  this  is 
unlikely,  then  we  need  SMEs  that  have  relevant  qualifications  and  relevant  experience 
in  their  recent  past,  e.g.  QL5  NCIOP  for  the  TS.  A  QL4/5  NCIOP  (or  a  QL3  NCIOP, 
with  significant  experience)  for  the  RTl.  Equivalent  needs  apply  to  the  SWC  SME. 
Whoever  is  provided,  they  need  to  stay  for  the  duration  of  the  trial  (two  days  for  the 
pilot,  five  days  for  the  study)  so  that  we  don't  have  to  repeat  the  start  up  briefing  / 
learning  time  on  subsequent  days. 

Subjects;  For  the  pilot  smdy,  two  OROs  will  be  needed  for  a  half  day  each.  For  the 
actual  study,  a  further  eight  OROs  (each  for  half  a  day)  must  have  completed  the  ORO 
course  and  have  had  at  least  one  year  as  an  ORO  in  a  HALIFAX  class  frigate,  as 
recently  as  possible,  but  certainly  within  the  last  four  years. 

5.  HSf  personnel:  The  make-up  of  the  T&E  team  for  the  Pilot  Trial  is  largely  dependent 
on  whether  software  fixes  6.1.1  and  6. 1 .2  are  in  place.  The  following  table  outlines  the 
team  required,  depending  whether  such  fixes  are  done  or  not. 


T&E 

Personnel 

Software 

Software 

function 

Required 

fixed 

not  fixed 

Entity  controller-background 

HSl®  staff  -  trained 

NO 

YES 

Entity  controller-threat 

HSI®  staff  -  trained 

NO 

YES 

Information  provider 

HSl®  -  Navy  SME 

YES 

YES 

ORO  Observer 

HSI®  -Navy  SME 

YES 

YES 

Trial  co-ordinator  (also 

HSl®  senior  staff 

YES  (also  controls 

YES 

controls  some  dynamic 

some  dynamic 

entities 

entities) 

1 

TOTAL 

3 

5 

6.4  Scenario  modification 

To  ensure  appropriate  distribution  of  ORO  attention,  some  additional  tasks  will  need  to  be 
provided.  We  are  working  out  details  of  these  and  how  they  may  be  practically  implemented, 

6.5  Other  issues 

We  recommend  an  early  discussion  with  DREA  and  LMC  to  further  elaborate  upon  the  specific 
requirements  for  the  development  of  "test"  tracks  that  will  address  existing  radar  technology 
problems  in  track  detection  and  maintenance  and  other  track  contexts  that  are  likely  to  be 
benefited  by  MSDF  improvements. 
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Annex  A;  Scenario  Description 

General  Situation 

As  a  result  of  Yemen’s  continued  refusal  to  take  decisive  action  against  Al  Qaeda  cells  operating 
within  its  borders,  President  Bush  has  included  Yemen  as  an  element  in  the  Axis  of  Evil.  The  USS 
George  Washington  Battle  Group  has  been  positioned  in  the  Gulf  of  Aden  and  carrier  based  aircraft 
have  been  conducting  overt  operations  to  demonstrate  air  superiority  in  the  Gulf  up  to  the  territorial 
air  space  of  Yemen.  President  Bush  has  been  working  to  build  international  support  for  military 
action  against  Yemen  but  has  met  with  resistance,  particularly  from  the  Arab  nations,  Russia  and 
China. 

While  coalition-building  efforts  continue  naval  forces  from  the  United  States,  Canada,  the  United 
Kingdom  and  Australia  have  begun  to  monitor  shipping  in  the  Gulf  of  Aden  and  the  Red  Sea  in 
anticipation  of  a  UN  resolution  establishing  a  Maritime  Interdiction  Operation  in  the  region.  The 
main  focus  of  the  monitoring  effort  is  in  the  Gulf  of  Aden  as  concerns  mount  that  ships  from  Iran 
may  be  transporting  Al  Qaeda  supplies  and  personnel  to  Yemen  ports.  HALIFAX  is  the  only  ship 
in  the  southern  portion  of  the  Red  Sea.  HALIFAX’S  role  for  the  time  being  is  to  become  familiar 
with  the  maritime  environment  in  the  area.  If  a  MIO  is  authorized  by  the  UN,  HALIFAX  will 
participate. 

Yemen  has  claimed  that  its  forces  have  been  preparing  for  strikes  against  the  Al  Qaeda  cells  but 
there  appears  to  be  little  resolve  for  action.  Al  Qaeda  members  are  believed  to  be  relatively  free  to 
travel  within  Yemen,  and  are  known  to  cross  borders  into  southern  Saudi  Arabia,  Eritrea  and 
Ethiopia.  Refusing  to  be  intimidated,  Yemen  military  aircraft  have  on  occasion  intercepted  US 
Navy  aircraft  in  international  airspace  and  have  approached  the  George  Washington  to  within  20 
miles.  They  have  also  routinely  approached  HAL  to  within  5  miles.  To  date  the  Yemen  aircraft 
have  carried  only  air-to-air  missiles;  they  have  not  carried  bombs  or  anti-ship  missiles.  Yemen 
aircraft  include  Fitters  (FBA),  Fishbeds  (intercept)  and  Mirage  (anti-ship).  The  Fitters  carry  two 
5001b  iron  bombs  and  the  Mirage  carry  two  Exocet  ASMs.  Yemen  Tarantual  patrol  boats  have 
routinely  operated  within  territorial  waters  in  the  Gulf  of  Aden  and  Red  Sea.  The  Tarantuals  carry 
SSN-2C  anti-ship  missiles.  There  are  two  operational  coastal  missile  batteries  on  the  Red  Sea  coast 
and  their  associated  Puff  Ball  radars  are  routinely  active  for  extended  periods  of  time.  Tension 
between  Yemen  and  the  US  has  been  relatively  high  since  the  attack  on  the  USS  COLE  in  Aden, 
with  the  September  1 1*  attacks  serving  to  heighten  emotions  even  more.  Although  the  Yemen 
government  has  made  no  threats,  Al  Qaeda  elements  within  Yemen  have  made  unspecified  threats 
against  American  interests  in  the  area. 

To  further  complicate  the  situation,  Yemen  and  Eritrea  have  begun  a  war  of  words  over  oil  rights  in 
the  southern  Red  Sea.  American  oil  companies  based  in  Eritrea  operate  four  oil  rigs  in  international 
water  just  off  of  the  Yemen  island  of  Kubra.  Yemen  claims  that  the  oil  rigs  are  within  their 
territorial  waters  but  the  international  community  recognizes  the  area  as  international  waters. 

Yemen  maintains  a  naval  presence  in  the  vicinity  of  the  oil  rigs,  usually  consisting  of  one  or  two 
Tarantual  patrol  boats.  Yemen  aircraft  on  occasion  approach  Eriteran  airspace  to  exert  pressure  and 
probe  defences. 

HALIFAX  is  operating  near  the  major  shipping  lane  between  the  Suez  Canal  and  the  Gulf  region. 
Large  ships  including  oil  tankers  regularly  pass  through  the  Bab  el  Mandeb,  the  strait  between 
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Yemen  and  Eritrea/Djibouti.  Other  maritime  traffic  in  the  area  consists  of  some  small  wooden 
fishermen  and  pleasure  craft,  supply  boats  going  to  and  from  the  oil  rigs  from  Aseb,  Eritrea,  and 
small,  fast  cigarette  boat-like  smuggler  traffic  between  Yemen  and  Eritrea.  TTiese  smugglers  trade 
primarily  in  cigarettes  and  liquor,  but  intelligence  indicated  that  they  may  also  be  transporting  A1 
Qaeda  personnel.  The  waters  in  the  area  are  very  polluted,  with  floating  animal  carcasses  and  other 
debris  being  commonplace. 

Apart  from  the  routine  commercial  air  traffic  in  the  region,  helicopters  fly  between  Aseb,  Eritrea 
and  the  oil  rigs,  and  numerous  small  private  planes  transit  between  Eritrea,  Yemen  and  Saudi 
Arabia.  Many  of  these  planes  are  flown  by  rich  Saudi's  ,  the  same  demographic  that  sails 
recreational  boats  in  the  area. 

HMCS  HALIFAX  is  carrying  one  Sea  King  helicopter  and  has  a  full  operational  weapons  load.  The 
ship  is  manned  at  Remar  and  has  no  major  operational  defects.  The  ship  completed  a  full  RSP, 
including  Workups,  three  weeks  prior  to  deploying  from  Halifax.  The  ship  has  been  on  patrol  for 
eight  days  and  expects  to  remain  another  12  prior  to  being  relieved.  HALIFAX  will  continue  to 
rotate  into  the  patrol  area  for  the  foreseeable  future.  HALIFAX  is  operating  under  national  OPCON 
and  is  using  Canadian  OPTASKS  and  ROE.  ROE  include  the  authority  to  warn  off  aircraft  up  to 
and  including  Warning  5.  HALIFAX  is  in  the  Second  Degree  of  Readiness,  with  CCS  is  in  Semi- 
Auto,  STIRS  in  Hot  Standby,  and  all  missiles  tuned. 

T  time  is  1230  local.  Port  watch  is  just  coming  on  watch  for  the  afternoon.  Skies  are  clear  and 
visibility  is  good.  The  captain  is  on  the  bridge.  The  XO  is  in  her  cabin. 

Rules  of  Engagement: 

Self  defence  only.  Use  of  FC  radars  for  height  finding  is  authorized. 

identification  Criteria: 

AIR; 

Hostile; 

In  process  of  conducting  a  hostile  act 
Suspect: 

Contact  that  has  previously  committed  a  hostile  act  but  is  no  longer  on  an  attack  profile 
Contact  that  displays  suspicious  or  potentially  threatening  behaviour 
Any  Yemen  military  aircraft 

Neutral 

Any  non-military  aircraft  acting  in  a  non-threatening  manner 
Military  aircraft  belonging  to  a  neutral  nation 

Assumed  Friend 

Single  contact  that  correlates  within  5  degrees  of  an  ESM  bearing  of  a  friendly  electronic  emission. 
Contact  on  same  bearing  as  another  showing  mode  4 

Friend 

Visually  sighted  and  recognized  as  a  friendly  military  aircraft 

Contact  displaying  a  valid  mode  4  IFF  reply  and  is  the  only  contact  on  that  bearing 
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Unknown 

Evaluated  track  that  cannot  be  identified 
SURFACE: 

Hostile 

In  process  of  conducting  a  hostile  act 

Suspect 

Contact  that  has  previously  committed  a  hostile  act  but  is  uo  longer  on  an  attack  profile 
Contact  that  displays  suspicious  or  potentially  threatening  behaviour 
Any  Yemen  military  vessel 

Neutral 

Any  non-military  vessel  acting  in  a  non-threatening  manner 
Military  vessel  belonging  to  a  neutral  nation 

Friend 

Contact  correlates  with  friendly  ESM 
Contact  visually  identified  as  friend 
Contact  with  proper  mode  4  response 

Unknown 

Evaluated  track  that  cannot  be  identified 


Yemen  Order  of  Battle 


Platform 

Weapons 

Emitters 

Surface’ 

3  X  TarantuI 

4XSSN-2C 

1  X76mm 

2  X  30mm  CiWS 

Search  -  Plank  Shave 

Fire  Control  -  Bass  Tilt 

2  X  Coastal  Missile  Batteries 

8  X  SSN-2C  each 

Puff  Ball 

AIR 

12XMiraQe 

Exocet 

Cyrano 

22  X  Fitter  (FBA) 

2  X  500lb  bombs 

High  Fix 

8  X  Fishbed  (Intercept) 

Air  to  air 

Jay  Bird 

Eritrea  Order  of  Battle 


Platform 

Weappn_s 

Emitters 

AIR 

22  X  Fitter  (FBA) 

2  X  500lb  bombs 

High  Fix 
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